Deskewing by space-variant deblurring

Skew and motion blur are significant challenges when camera and scene of interest are in two different media. Skew occurs due to spatially varying refraction on a dynamic water surface, whereas motion blur results from multiple intensities impinging on the imaging sensor during camera exposure time due to time varying refraction. In this paper, we propose a technique to restore underwater images degraded by attenuated water waves. Following others [4] [2], we also assume that the static camera is looking vertically downwards and imaging a planar scene but through spatially decaying periodic water waves (such as waves due to breeze in shallow water bodies). Also, we allow for the attenuation factor as well as the direction of the water waves to undergo changes during the exposure time of the camera (a situation not handled by the state-ofthe-art work in [3]). We propose a shot detection method that automatically segments a captured video into groups of frames wherein each group is governed by a dominant water wave direction (i.e. unidirectional) and a single exponential factor of attenuation. Within each segment, we average the frames and show that the blur induced at different pixel locations are scaled versions of each other and model these scale factors through a virtual depth map. Note that despite the scene being planar, the blur induced due to attenuated water waves is space-variant in nature. We pose deskewing of the planar scene as equivalent to a space-variant deblurring problem corresponding to a 3D scene with depth profile being the same as that of the virtual depth map. We propose an alternating framework to solve for the latent image from a single blurred observation. The procedure can likewise be repeated for other segments too, if needed.